INFORMATION RETRIEVAL

Biomedical Document Retrieval for Clinical Decision Support System (P18-3)

Challenge:	Named Entity Recognition (NER) is a useful component in NLP applications.
Approach:	They propose to use annotated named entity corpora to classify a given entity into a category within a textual document.
Outcome:	The proposed model achieves an F1 score of 0.80 on an unseen dataset for Indian languages.

Copied to clipboard

Jainisha Sankhavara

Unification-based Reconstruction of Multi-hop Explanations for Science Questions (2021.eacl-main)

Challenge:	Thousands of articles are being added into biomedical literature each year.
Approach:	They compare statistical and NLP based approaches for biomedical document retrieval . they model biomedically document retrievals as a learning to rank problem .
Outcome:	The proposed approach is based on statistical and NLP methods and will be applied to biomedical document retrieval and ranking systems.

Copied to clipboard

Marco Valentino, Mokanarangan Thayaparan, André Freitas

Explainability & Interpretability in NLP

Challenge:	Existing approaches build explanations considering each question in isolation, but new approach leverages explanatory patterns emerging in scientific explanations.
Approach:	They propose a framework for reconstructing multi-hop explanations in science Question Answering . they integrate lexical relevance with the notion of unification power to rank atomic facts .
Outcome:	The proposed method achieves results competitive with Transformers, but is faster and scalable to large explanatory corpora.

Training a Ranking Function for Open-Domain Question Answering (N18-4)

Copied to clipboard

Phu Mon Htut, Samuel Bowman, Kyunghyun Cho

Question Answering

Challenge:	Recent advances in machine reading have inspired researchers to combine Information Retrieval with machine reading to tackle open-domain QA.
Approach:	They propose two neural network rankers that assign scores to different passages based on their likelihood of containing the answer to a given question.
Outcome:	The proposed models achieve human level performance in open-domain QA compared to reading comprehension-style QA because it is difficult to retrieve the pieces of paragraphs that contain the answer to the question.

Agent Assist through Conversation Analysis (2020.emnlp-demos)

Copied to clipboard

Kshitij Fadnis, Nathaniel Mills, Jatin Ganhotra, Haggai Roitman, Gaurav Pandey, Doron Cohen, Yosi Mass, Shai Erera, Chulaka Gunasekara, Danish Contractor, Siva Patel, Q. Vera Liao, Sachindra Joshi, Luis Lastras, David Konopnicki

Natural Language Interfaces

Challenge:	Using conversational approach to information retrieval for agent assistance, customer support agents are a critical part of an organization's customer support team.
Approach:	They propose a conversational approach to information retrieval for agent assistance that monitors an evolving conversation and recommends both responses and URLs of documents.
Outcome:	The proposed system monitors an evolving conversation and recommends both responses and URLs of documents the agent can use in replies to their client.

Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association (P18-4)

Copied to clipboard

Yafang Huang, Zuchao Li, Zhuosheng Zhang, Hai Zhao

Machine Translation

Challenge:	a pinyin input method engine (IME) allows users to input Chinese into a computer by typing pinyan through the common keyboard.
Approach:	They present a pinyin IME that integrates neural machine translation and IR to offer amusive and customizable association ability.
Outcome:	The Moon IME integrates neural machine translation and IR to offer amusive association ability.

KIMERA: Injecting Domain Knowledge into Vacant Transformer Heads (2022.lrec-1)

Copied to clipboard

Benjamin Winter, Alexei Figueroa Rosero, Alexander Löser, Felix Alexander Gers, Amy Siu

Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning (2021.eacl-main)

Challenge:	Recent studies show that transformer models lack specific domain knowledge and are under-performing in broad domains like the medical domain.
Approach:	They propose a method for retraining and instilling attention heads with structured domain knowledge by masking redundant attention heads.
Outcome:	The proposed method improves on seven datasets in the medical domain in information retrieval and clinical outcome prediction settings.

Copied to clipboard

Alon Jacovi, Gang Niu, Yoav Goldberg, Masashi Sugiyama

Statistical Foundations of DIME: Risk Estimation for Practical Index Selection (2026.eacl-short)

Challenge:	Current methods for document set expansion for large collections are based on word-frequency or bag-of-words document similarity metrics.
Approach:	They propose to extend the IR approach by treating the problem as an instance of positive-unlabeled (PU) learning . they propose solutions for each challenge and empirically validate them with ablation tests .
Outcome:	The proposed method improves on a PubMed abstract retrieval task . it is compared with existing methods and empirically validated with ablation tests .

Copied to clipboard

Giulio D'Erasmo, Cesare Campagnano, Antonio Mallia, Pierpaolo Brutti, Nicola Tonellotto, Fabrizio Silvestri

TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation (P18-1)

Challenge:	High-dimensional dense embeddings are noisy or redundant, causing performance degradation and causing errors.
Approach:	They propose a method that scores each dimension by fusing the embeddings into a query-dependent matrix.
Outcome:	The proposed method improves retrieval effectiveness and reduces embedding size by an average 50% of across different models and datasets at inference time.

Copied to clipboard

Alexander Fabbri, Irene Li, Prawat Trairatvorakul, Yijiao He, Weitai Ting, Robert Tung, Caitlin Westerfield, Dragomir Radev

GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration (2023.acl-demo)

Challenge:	TutorialBank is a publicly available dataset that aims to facilitate NLP education and research . a google search of "Natural Language Processing" returns over 100 million hits with papers, tutorials, 1 http://aan.how blog posts, codebases and other related online resources.
Approach:	They have manually collected and categorized over 5,600 resources on NLP . they have created a search engine and command-line tool to search the corpus .
Outcome:	The tutorial bank dataset is the largest manually-picked corpus of resources intended for NLP education . it includes lists of research topics, relevant resources for each topic, prerequisite relations among topics .

Copied to clipboard

Aleksandra Piktus, Odunayo Ogundepo, Christopher Akiki, Akintunde Oladipo, Xinyu Zhang, Hailey Schoelkopf, Stella Biderman, Martin Potthast, Jimmy Lin

BioMedBERT: A Pre-trained Biomedical Language Model for QA and IR (2020.coling-main)

Challenge:	Using the mature and well-tested methods from the domain of Information Retrieval (IR) we propose to integrate Pyserini with Hugging Face to provide qualitative analysis tools for NLP researchers.
Approach:	They propose to integrate Pyserini with Hugging Face to provide qualitative analysis tools for NLP researchers.
Outcome:	The proposed tools can be integrated with the Hugging Face ecosystem of open-source AI libraries and artifacts.

Copied to clipboard

Souradip Chakraborty, Ekaba Bisong, Shweta Bhatt, Thomas Wagner, Riley Elliott, Francesco Mosconi

Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval (2024.emnlp-main)

Challenge:	SARS-CoV-2 pandemic highlighted importance of moving quickly with biomedical research.
Approach:	They propose a textual data mining tool that supports literature search to accelerate the work of researchers in the biomedical domain.
Outcome:	The proposed model achieves state-of-the-art results on the QA fine-tuning task on BioASQ 5b, 6b and 7b datasets.

Copied to clipboard

Tianyi Hu, Maria Maistro, Daniel Hershcovich

Building an Efficient Multilingual Non-Profit IR System for the Islamic Domain Leveraging Multiprocessing Design in Rust (2024.emnlp-industry)

Challenge:	Adapting recipes to cultural differences presents significant importance and challenges . bridging cultural differences is a challenge, but IR can help.
Approach:	They propose a framework that preserves the original recipe and its cultural appropriateness for the target culture.
Outcome:	The proposed framework preserves the original recipe and its cultural appropriateness for the target culture while maintaining relevance to the original.

Copied to clipboard

Vera Pavlova, Mohammed Makhlouf

The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes (2021.acl-short)

Challenge:	Existing models that are pre-trained on a general domain can deteriorate performance due to domain shift when applied to new domains.
Approach:	They propose to train a multilingual non-profit IR system for the Islamic domain using Rust Language capabilities.
Outcome:	The proposed model outperforms models pre-trained on general domains and on resource-constrained devices.

Copied to clipboard

Nils Reimers, Iryna Gurevych

IR like a SIR: Sense-enhanced Information Retrieval for Multiple Languages (2021.emnlp-main)

Challenge:	Existing studies have shown that dense representations outperform sparse representations with large index sizes.
Approach:	They propose to use dense low-dimensional representations to retrieve relevant documents . they show performance decreases quicker for increasing index sizes than for sparse representations .
Outcome:	The proposed representations outperform sparse representations with large index sizes.

Copied to clipboard

Rexhina Blloshmi, Tommaso Pasini, Niccolò Campolungo, Somnath Banerjee, Roberto Navigli, Gabriella Pasi

Exploring efficient zero-shot synthetic dataset generation for Information Retrieval (2024.findings-eacl)

Challenge:	Recent advances in contextualized embeddings have made ranking on non-English documents cumbersome . a novel multilingual query expansion mechanism provides sense definitions as additional semantic information for the query.
Approach:	They propose a multilingual query expansion mechanism that leverages word sense information to enhance the model's performance.
Outcome:	The proposed model performs better than its supervised and unsupervised alternatives across languages while being trained on English Robust04 data.

Copied to clipboard

Tiago Almeida, Sérgio Matos

Responsible & Trustworthy NLP

Challenge:	Recent advances in large language models offer a new avenue of generating synthetic training data to train neural retrieval models for unlabelled data collections.
Approach:	They propose a method to generate high-quality synthetic datasets using a small language model and a filtering mechanism to ensure the quality of generated questions.
Outcome:	The proposed method outperforms unsupervised retrieval methods such as BM25 and pretrained monoT5.

Unified Multimodal Interleaved Document Representation for Retrieval (2026.findings-eacl)

Copied to clipboard

Jaewoo Lee, Joonho Ko, Jinheon Baek, Soyeong Jeong, Sung Ju Hwang

Multimodality

Challenge:	Existing methods focus on textual content, ignoring the fact that documents can contain multiple modalities.
Approach:	They propose a method that holistically embeds documents interleaved with multiple modalities . they use vision-language models that combine text, images, and tables into a unified format .
Outcome:	The proposed method outperforms baselines on textual and multimodal queries.

Intent Recognition in Doctor-Patient Interviews (2020.lrec-1)

Copied to clipboard

Robin Rojowiec, Benjamin Roth, Maximilian Fink

Sentiment Analysis

Challenge:	Currently, up to 20 percent of patients are misdiagnosed in medical training programs.
Approach:	They propose to annotate doctor-patient interviews with intent inventory and information retrieval methods that are robust with respect to small amounts of training data.
Outcome:	The proposed models provide baseline performance scores on the data set for further research.

DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management (2025.findings-emnlp)

Copied to clipboard

Kai Yin, Xiangjue Dong, Chengkai Liu, Lipai Huang, Yiming Xiao, Zhewei Liu, Ali Mostafavi, James Caverlee

An LCF-IDF Document Representation Model Applied to Long Document Classification (2024.lrec-main)

Challenge:	Existing information retrieval benchmarks focus on general or specialized domains, such as medicine or finance, neglecting the unique linguistic complexity and diverse information needs encountered in disaster management scenarios.
Approach:	DisastIR is the first comprehensive IR evaluation benchmark specifically tailored for disaster management.
Outcome:	DisastIR covers 48 retrieval tasks derived from six search intents and eight general disaster categories . evaluations show no single model excelling universally .

Copied to clipboard

Renzo Arturo Alva Principe, Nicola Chiarini, Marco Viviani

Which Works Best for Vietnamese? A Practical Study of Information Retrieval Methods across Domains (2026.findings-eacl)

Challenge:	Document representation models have been used for years in NLP and Text Mining tasks but are limited when it comes to capturing the deeper semantics and context of textual data.
Approach:	They propose to use a Latent Concept Frequency-Inverse Document Frequence model to exploit the advantages of TF-IDF while incorporating semantic context into the model.
Outcome:	The proposed model outperforms existing models on the Long Document Classification task and shows that it performs better than TF-IDF and BERT-like representation models.

Copied to clipboard

Long S. T. Nguyen, Tho T. Quan

Analysis of Automated Document Relevance Annotation for Information Retrieval in Oil and Gas Industry (2025.emnlp-industry)

Challenge:	Existing studies on Large Language Models (LLMs) are limited to single domains or curated datasets.
Approach:	They propose a domain-normalized, multi-domain benchmark for Vietnamese IR . they evaluate lexical, neural-sparse, late-interaction, dense, and hybrid paradigms .
Outcome:	The proposed benchmarks cover six domains and ten datasets across education, legal, healthcare, customer support, lifestyle reviews, and open-domain knowledge.

Copied to clipboard

João Vitor Mariano Correia, Murilo Missano Bell, João Vitor Robiatti Amorim, Jonas Queiroz, Daniel Pedronette, Ivan Rizzo Guilherme, Felipe Lima de Oliveira

Beyond [CLS] through Ranking by Generation (2020.emnlp-main)

Challenge:	Lack of high-quality test collections challenges Information Retrieval (IR) in specialized domains.
Approach:	They compare supervised classifiers against zero-shot Large Language Models for automated relevance annotation in the oil and gas industry using human expert judgments as a benchmark.
Outcome:	The proposed classifier outperforms LLMs in the oil and gas industry using human expert judgments.

Copied to clipboard

Cicero Nogueira dos Santos, Xiaofei Ma, Ramesh Nallapati, Zhiheng Huang, Bing Xiang

NevIR: Negation in Neural Information Retrieval (2024.eacl-long)

Challenge:	Recent work on generative ranking models for Information Retrieval has focused on discriminative methods that learn a similarity function to compare questions and candidates answers.
Approach:	They propose to use a language model to train a ranking function that model the semantic similarity of documents and queries instead of discriminative ranking functions.
Outcome:	The proposed approaches are as effective as state-of-the-art discriminative models for the answer selection task and show unlikelihood losses are reduced for IR.

Copied to clipboard

Orion Weller, Dawn Lawrie, Benjamin Van Durme

Incorporating Multimodal Information in Open-Domain Web Keyphrase Extraction (2020.emnlp-main)

Challenge:	Negation is a common everyday phenomenon and has been a consistent area of weakness for language models.
Approach:	They ask IR models to rank two documents that differ only by negation . they find that most current information retrieval models do not consider negation.
Outcome:	The proposed benchmarks show that most current models do not consider negation . the results are similar to those found in the literature, but are poorer than random ranking .

Copied to clipboard

Yansen Wang, Zhen Fan, Carolyn Rose

Denoising Attention for Query-aware User Modeling (2024.findings-naacl)

Challenge:	Open-domain Keyphrase extraction (KPE) is a fundamental yet complex NLP task . effective designs encode within layout and formatting signals that point to where the important information can be found.
Approach:	They propose a multi-modal approach to open-domain keyphrase extraction (KPE) on the Web that leverages layout and formatting signals to aid in the task.
Outcome:	The proposed model outperforms state-of-the-art models on the open-domain keyphrase extraction task.

Copied to clipboard

Elias Bassani, Pranav Kasela, Gabriella Pasi

Challenge:	Recent work has proposed to build user models at query time by leveraging the Attention mechanism, which allows weighing the contribution of the user-related information w.r.t. the current query.
Approach:	They propose to use the Attention mechanism to build user models at query time by weighing the contribution of the user-related information w.r.t. the Attention variant adopts a robust normalization scheme and introduces . filtering mechanism to better discern among the user related data those helpful for personalization.
Outcome:	The proposed approach improves MAP, MRR, and NDCG above 15% w.r.t. other Attention variants at the state-of-the-art.

Negative Sampling Techniques in Dense Retrieval: A Survey (2026.findings-eacl)

Copied to clipboard

Laurin Wischounig, Abdelrahman Abdallah, Adam Jatowt

GUMsley: Evaluating Entity Salience in Summarization for 12 English Genres (2024.eacl-long)

Challenge:	Information Retrieval (IR) is fundamental to many modern NLP applications.
Approach:	They propose a taxonomy that categorizes negative sampling techniques in dense IR . they analyze them with respect to trade-offs between effectiveness, computational cost, implementation difficulty .
Outcome:	The proposed taxonomy categorizes techniques using random, static/dynamically mined, and synthetic datasets.

Copied to clipboard

Jessica Lin, Amir Zeldes

BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Language (2024.lrec-main)

Challenge:	Existing work on salient entity extraction relies on crowdsourcing or user statistics to derive labels for entities.
Approach:	They propose a dataset that defines salience using human summaries and shows high agreement between annotations based on whether a source entity is mentioned in the summary.
Outcome:	The proposed dataset shows that pre-trained models and zero-shot LLM prompting fail to capture salient entities in generated summaries.

Copied to clipboard

Konrad Wojtasik, Kacper Wołowiec, Vadim Shishkin, Arkadiusz Janz, Maciej Piasecki

CoRanking: Collaborative Ranking with Small and Large Ranking Agents (2025.findings-emnlp)

Challenge:	Existing multilingual evaluation benchmarks focus on IR in the Polish language, but the Polish is a relatively new field due to the limited availability of Polish datasets.
Approach:	They propose to establish large-scale resources for IR in the Polish language and translate them into a new benchmark which includes 13 datasets.
Outcome:	The proposed benchmarks are based on 13 open IR datasets in Polish and are a pioneering development in this area.

Copied to clipboard

Wenhan Liu, Xinyu Ma, Yutao Zhu, Lixin Su, Shuaiqiang Wang, Dawei Yin, Zhicheng Dou

Responsible & Trustworthy NLP

Challenge:	Listwise ranking based on Large Language Models (LLMs) has achieved state-of-the-art performance in Information Retrieval (IR) however, their effectiveness often depends on LLMs with massive parameter scales and computationally expensive sliding window processing, leading to substantial efficiency bottlenecks.
Approach:	They propose a Collaborative Ranking framework (CoRanking) for LLM-based listwise ranking based on large language models with massive parameter scales and computationally expensive sliding window processing.
Outcome:	The proposed framework reduces ranking latency by approximately 70% while improving effectiveness compared to the standalone large reranker.

MrRank: Improving Question Answering Retrieval System through Multi-Result Ranking Model (2024.findings-acl)

Copied to clipboard

Danupat Khamnuansin, Tawunrat Chalothorn, Ekapol Chuangsuwanich

Natural Language Interfaces

Challenge:	Existing IR techniques contain deficiencies, posing a performance bottleneck . combining diverse approaches to retrieve information is a viable strategy .
Approach:	They propose an approach that leverages learning-to-rank techniques to combine heterogeneous IR systems.
Outcome:	The proposed method outperforms existing methods on two Retrieval Question Answering tasks.

RoD-TAL: A Benchmark for Answering Questions in Romanian Driving License Exams (2026.findings-eacl)

Copied to clipboard

Andrei Vlad Man, Răzvan-Alexandru Smădu, Cristian-George Craciun, Dumitru-Clementin Cercel, Florin Pop, Mihaela-Claudia Cercel

Multimodality

Challenge:	a growing need for tools that support legal education, especially in under-resourced languages such as Romanian . we evaluate the capabilities of large language models and vision-language models in legal education .
Approach:	They evaluate the capabilities of Large Language Models and Vision-Language Models in Romanian driving law through textual and visual question-answering tasks.
Outcome:	The proposed model improves retrieval performance and QA accuracy in Romanian driving tests.

CMNEE:A Large-Scale Document-Level Event Extraction Dataset Based on Open-Source Chinese Military News (2024.lrec-main)

Copied to clipboard

Mengna Zhu, Zijie Xu, Kaisheng Zeng, Kaiming Xiao, Mao Wang, Wenjun Ke, Hongbin Huang

Collective Relevance Labeling for Passage Retrieval (2022.naacl-main)

Challenge:	Current research focuses on the general news or financial domains, with relatively few studies for military domain.
Approach:	They propose to annotate Chinese military news events from documents using a schema for the military domain.
Outcome:	The proposed dataset is large-scale, document-level open-source for the military domain . it contains 17,000 documents and 29,223 events, which are all manually annotated .

Copied to clipboard

Jihyuk Kim, Minsoo Kim, Seung-won Hwang

Redefining Absent Keyphrases and their Effect on Retrieval Effectiveness (2021.naacl-main)

Challenge:	Existing approaches to improve IR labels are incomplete and require computational overheads.
Approach:	They propose to distill knowledge for informed labeling without high computation overheads at evaluation time.
Outcome:	The proposed model outperforms state-of-the-art models while distilling the rankings better.

Copied to clipboard

Florian Boudin, Ygor Gallina

Challenge Dataset of Cognates and False Friend Pairs from Indian Languages (2020.lrec-1)

Challenge:	Neural keyphrase generation models can output absent keyphrases, which are keyphrase that do not appear in the source text.
Approach:	They propose a finer-grained categorization scheme that sheds more light on the impact of absent keyphrases on scientific document retrieval.
Outcome:	The proposed model shows that only 20% of the words that make up keyphrases actually serve as document expansion, but this small fraction behind much of the gains observed in retrieval effectiveness.

Copied to clipboard

Diptesh Kanojia, Malhar Kulkarni, Pushpak Bhattacharyya, Gholamreza Haffari

Multilinguality

Challenge:	Cognates are words that have a common etymological origin and can facilitate the Second Language Acquisition (SLA) however, they also pose a challenge to various NLP applications such as Machine Translation and Cross-lingual Sense Disambiguation.
Approach:	They create two cognate datasets for twelve Indian languages and use them to generate cognate sets.
Outcome:	The proposed datasets are curated using previously available baseline cognate detection approaches and evaluated with the help of lexicographers.

Redefining Retrieval Evaluation in the Era of LLMs (2026.eacl-long)

Copied to clipboard

Giovanni Trappolini, Florin Cuconasu, Simone Filice, Yoelle Maarek, Fabrizio Silvestri

GBT: Generative Boosting Training Approach for Paraphrase Identification (2023.findings-emnlp)

Challenge:	Traditional IR metrics assume that humans examine documents sequentially with diminishing attention to lower ranks.
Approach:	They propose a utility-based annotation schema that quantifies positive contribution of relevant passages and negative impact of distracting ones.
Outcome:	The proposed metric improves correlation with the end-to-end answer accuracy by up to 36% compared to traditional metrics.

Copied to clipboard

Rui Peng, Zhiling Jin, Yu Hong

Text Generation

Challenge:	Paraphrase Identification (PI) is a fundamental natural language understanding task with non-trivial challenges.
Approach:	They propose a Generative Boosting Training approach for Paraphrase Identification (PI) they use a seq2seq model to perform DA on misclassified instances periodically .
Outcome:	The proposed method outperforms state-of-the-art PI models on English and Chinese PI tasks with good efficiency and effectiveness.

Refocusing on Relevance: Personalization in NLG (2021.emnlp-main)

Copied to clipboard

Shiran Dudy, Steven Bedrick, Bonnie Webber

Text Generation

Challenge:	a number of negative effects exist when NLG systems are not grounded to a specific input text.
Approach:	They argue that NLG systems should focus on making use of additional context . they argue that relevance should be thought of as a crucial tool for user-oriented text-generating tasks .
Outcome:	The proposed approach is more of the rule than the exception, the authors argue . they argue that value-sensitive design represents a crucial path forward .

HIL: Hybrid Isotropy Learning for Zero-shot Performance in Dense retrieval (2024.naacl-long)

Copied to clipboard

Jaeyoung Kim, Dohyeon Lee, Seung-won Hwang

Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence (2025.acl-long)

Challenge:	Recent advances in dense retrieval models have brought ColBERT to prominence in information retrieval, but it is underperforming in zero-shot tasks.
Approach:	They propose a Hybrid Isotropy Learning architecture that integrates isotropic and anisotropic representations to improve zero-shot retrieval performance.
Outcome:	The proposed model outperforms the baseline ColBERT model in BEIR benchmarks.

Copied to clipboard

Mohsen Fayyaz, Ali Modarressi, Hinrich Schuetze, Nanyun Peng

Pre-trained Language Models for Entity Blocking: A Reproducibility Study (2024.naacl-long)

Challenge:	Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer.
Approach:	They repurpose a relation extraction dataset to quantify the impact of heuristic biases on retrievers like Dragon+ and Contriever.
Outcome:	The proposed models exhibit catastrophic performance degradation when multiple biases combine, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer.

Copied to clipboard

Runhui Wang, Yongfeng Zhang

Semantic Equivalence Detection: Are Interrogatives Harder than Declaratives? (L18-1)

Challenge:	Entity Resolution (ER) is an essential task in data integration . state-of-the-art neural IR models that are based on large language models have not been evaluated on the ER task.
Approach:	They evaluate state-of-the-art neural IR models that are based on large language models on a wide range of real-world datasets and evaluate their generalization abilities.
Outcome:	The proposed methods have been evaluated on a wide range of datasets and their generalization abilities.

Copied to clipboard

João Rodrigues, Chakaveh Saedi, António Branco, João Silva

Challenge:	Semantic Text Similarity (STS) tasks are often not seen as similar to semantic equivalence detection tasks.
Approach:	They propose to assess the performance of different approaches to STS over different types of textual segments.
Outcome:	The proposed methods differ in performance over different types of textual segments, including declaratives and interrogatives, under conditions of comparability.

Tracking Textual Similarities in Neo-Latin Drama Networks (2022.lrec-1)

Copied to clipboard

Andrea Peverelli, Marieke van Erp, Jan Bloemendal

FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions (2025.naacl-long)

Challenge:	a wide international network of exchanges between writers of different nationalities is characteristic of the early Modern Era.
Approach:	They describe the first experiments to track the inter-national network of text reuse within the Early Modern community of Neo-Latin humanists.
Outcome:	The results show that the early Modern writers and writers are part of a wider, universal, intellectual community.

Copied to clipboard

Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini

Back to the Basics: A Quantitative Analysis of Statistical and Graph-Based Term Weighting Schemes for Keyword Extraction (2021.emnlp-main)

Challenge:	Modern language models (LMs) are capable of following long and complex instructions that enable a large and diverse set of user requests.
Approach:	They propose a dataset that contains an instruction evaluation benchmark and a training set to help IR models learn to follow instructions.
Outcome:	The proposed model improves after fine-tuning on a training set and rigorous instruction evaluation benchmark.

Copied to clipboard

Asahi Ushio, Federico Liberatore, Jose Camacho-Collados

Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction (2024.findings-acl)

Challenge:	Term weighting schemes are widely used in Natural Language Processing and Information Retrieval.
Approach:	They perform an exhaustive and large-scale empirical comparison of term weighting methods in the context of keyword extraction using tf-idf.
Outcome:	The proposed methods have advantages over tf-idf, and qualitative differences between them.

Copied to clipboard

Albert Sawczyn, Katsiaryna Viarenich, Konrad Wojtasik, Aleksandra Domogała, Marcin Oleksy, Maciej Piasecki, Tomasz Kajdanowicz

Question Answering

Challenge:	Existing KBQA datasets are outdated and inefficient in human labor, and assisting tools like Large Language Models (LLM) are not utilized to reduce the workload.
Approach:	They propose a semi-automated question answering task that uses structured knowledge graphs to answer extensive knowledge-intensive questions.
Outcome:	The proposed approach includes KBQA, MRC, and Information Retrieval tasks for low-resource languages.

Efficient Long Context Language Model Retrieval with Compression (2025.acl-long)

Copied to clipboard

Minju Seo, Jinheon Baek, Seongyun Lee, Sung Ju Hwang

Measuring and Addressing Indexical Bias in Information Retrieval (2024.findings-acl)

Challenge:	Long Context Language Models (LCLMs) are a new paradigm for Information Retrieval . however, processing large number of passages within in-context for retrieval is computationally expensive . a proposed compression approach for LCLM retrieval maximizes retrieval performance while minimizing the length of the compressed passages.
Approach:	They propose a new compression approach tailored to maximize retrieval performance while minimizing the length of compressed passages.
Outcome:	The proposed approach maximizes retrieval performance while minimizing the length of compressed passages while reducing the in-context size by 1.91.

Copied to clipboard

Caleb Ziems, William Held, Jane Dwivedi-Yu, Diyi Yang

IR2: Information Regularization for Information Retrieval (2024.lrec-main)

Challenge:	Information Retrieval (IR) systems may not optimize rankings for fairness, neutrality, or the balance of ideas.
Approach:	They propose to use a framework to automatically audit IR rankings for indexical biases, or biase in the positional order of documents.
Outcome:	The proposed bias metric can help predict when and how indexical bias will shift a reader’s opinion.

Copied to clipboard

Jianyou Wang, Kaicheng Wang, Xiaoyue Wang, Weili Cao, Ramamohan Paturi, Leon Bergen

Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023.emnlp-main)

Challenge:	Effective information retrieval (IR) in settings with limited training data remains a challenging task.
Approach:	They propose a technique for reducing overfitting during synthetic data generation . they use DORIS-MAE, ArguAna, and WhatsThatBook as examples .
Outcome:	The proposed technique outperforms previous methods and reduces cost by 50% on three recent IR tasks characterized by complex queries.

Copied to clipboard

Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, Zhaochun Ren

CoIR: A Comprehensive Benchmark for Code Information Retrieval Models (2025.acl-long)

Challenge:	Existing work utilizes generative LLMs for Information Retrieval (IR) rather than direct passage ranking.
Approach:	They investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR and use a test set to verify the model’s ability to rank unknown knowledge.
Outcome:	The proposed model outperforms a 3B supervised model on the BEIR benchmark.

Copied to clipboard

Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Hao Zhang, Xinyi Dai, Yasheng Wang, Ruiming Tang